Automated Clustering and Program Repair for Introductory Programming Assignments

Providing feedback on programming assignments is a tedious task for the instructor, and even impossible in large MOOCs with thousands of students. In this paper, we present a novel technique for automatic feedback generation: (1) For a given programming assignment, we automatically cluster the correct student attempts based on their underlying strategy. From each class we select one student attempt as a specification for the given strategy. (2) Given an incorrect student attempt we run a repair procedure against all specifications, and automatically generate the minimal repair based on one of them. We evaluated the approach on 56,806 student attempts over 18 problems from an undergraduate programming course and a MOOC course. In our evaluation we first automatically clustered correct attempts and generated specifications from the correct student attempts and then used these specifications to repair the incorrect student attempts without any human intervention, finding repairs for 86% of incorrect student attempts in around 7 seconds per attempt.